# Visual Understanding

PromptPilot is an intelligent solution platform that focuses on optimizing large models and achieving user task intentions. Through interactive feedback, the platform can automatically optimize multi-step, multimodal, and multi-scenario tasks, providing users with efficient intelligent solutions, suitable for enterprises and individuals to improve work efficiency and task completion quality.

Liquid is an autoregressive generative model that facilitates seamless integration of visual understanding and text generation by decomposing images into discrete codes and sharing feature space with text tokens. The main advantage of this model is the elimination of the need for externally pre-trained visual embeddings, reducing resource dependence, while simultaneously discovering a synergistic effect between understanding and generation tasks through the law of scaling.

Kimi-VL is an advanced expert-mixed visual language model designed for multi-modal reasoning, long-context understanding, and powerful agent capabilities. This model excels in several complex domains, boasting efficient 2.8B parameters while exhibiting outstanding mathematical reasoning and image understanding capabilities. Kimi-VL sets a new standard for multi-modal models with its optimized computational performance and ability to handle long inputs.

UniTok is an innovative visual tokenization technology designed to bridge the gap between visual generation and understanding. Through multi-codebook quantization technology, it significantly improves the representation capability of discrete tokenizers, enabling them to capture richer visual details and semantic information. This technology breaks through the bottleneck of traditional tokenizers in the training process, providing an efficient and unified solution for visual generation and understanding tasks. UniTok excels in image generation and understanding tasks, such as achieving a significant zero-shot accuracy improvement on ImageNet. The main advantages of this technology include efficiency, flexibility, and strong support for multimodal tasks, bringing new possibilities to the field of visual generation and understanding.

ZeroBench is a benchmark specifically designed to evaluate the visual understanding capabilities of large multimodal models (LMMs). It challenges the limits of current models through 100 meticulously crafted and rigorously vetted complex questions, along with 334 sub-questions. This benchmark aims to address the shortcomings of existing visual benchmarks by offering a more challenging and high-quality evaluation tool. ZeroBench's primary strengths are its high difficulty, lightweight design, diversity, and high quality, enabling it to effectively differentiate model performance. Additionally, it provides detailed sub-question evaluation, helping researchers better understand the reasoning abilities of the models.

MiniCPM-o 2.6 is the latest and most powerful model in the MiniCPM-o series. Built upon SigLip-400M, Whisper-medium-300M, ChatTTS-200M, and Qwen2.5-7B, it boasts 8 billion parameters. It excels in visual understanding, speech interaction, and multimodal live broadcasting, supporting real-time voice conversations and diverse live streaming features. The model performs excellently in the open-source community, surpassing several well-known models. Its strengths include efficient inference speed, low latency, and minimal memory and power consumption, allowing for effective multimodal live streaming on devices such as iPads. Moreover, MiniCPM-o 2.6 is user-friendly, supporting multiple usage approaches including CPU inference with llama.cpp, quantized models in int4 and GGUF formats, and high-throughput inference with vLLM.

Qwen2-VL-72B is the latest iteration of the Qwen-VL model, reflecting nearly a year of innovative advancements. This model has achieved state-of-the-art performance in visual understanding benchmarks, including MathVista, DocVQA, RealWorldQA, MTVQA, and more. It can comprehend videos exceeding 20 minutes and can be integrated into devices such as smartphones and robots for automated operations based on visual contexts and text instructions. In addition to English and Chinese, Qwen2-VL now supports understanding textual content in various languages found in images, including most European languages, Japanese, Korean, Arabic, Vietnamese, and others. Model architecture updates include Naive Dynamic Resolution and Multimodal Rotary Position Embedding (M-ROPE), enhancing its multimodal processing capabilities.

GLM-4V-9B is a new generation pre-trained model released by Zhi谱AI. It supports 1120*1120 high-resolution English and Chinese multi-turn dialogues, as well as visual understanding capabilities. In multimodal evaluations, GLM-4V-9B demonstrates superior performance surpassing GPT-4-turbo-2024-04-09, Gemini 1.0 Pro, Qwen-VL-Max, and Claude 3 Opus.

Video-LLaVA is a model for learning joint visual representations by training through prefix projection alignment. It aligns video and image representations, leading to better visual understanding. The model boasts efficient learning and inference speeds, making it suitable for video processing and visual tasks.